Method of creating video effects by use of keyframes

ABSTRACT

A video effect is created with a video processing machine having a plurality of functions, each of which has a state that is selectively variable in response to change in an associated function parameter. The method comprises selecting a set of functions, specifying a starting keyframe containing a starting value for each of the parameters, specifying an ending keyframe containing ending values for the parameters associated with the functions of the selected set and vacancies for the other parameters associated with other functions, and specifying a number of video frames. The starting keyframe is used to place the machine in a starting state, in which each function has the status defined by the value of its associated parameter in the starting keyframe, and a first video frame is processed with the processing machine in the starting state. The starting keyframe, the ending keyframe and the number of video frames are used to calculate a next frame value for each of the parameters associated with a function in the selected set. The next frame values are used to place the machine in a next frame state, in which each function of the set has the status defined by the next frame value of its associated parameter, while maintaining unaltered the values of the parameters associated with the other functions, and the next video frame is processed with the machine in the next frame state.

This is a continuation of application Ser. No. 08/055,109 filed May 3,1993, and now abandoned.

BACKGROUND OF THE INVENTION

This invention relates to a method of creating video effects by use ofkeyframes.

Two machines that are widely used in creating video effects are theproduction switcher and the digital video effects device. Each of thesevideo processing machines is used to modify an input video signal in acontrolled fashion in order to produce an output video signal. Themodification is commonly called an effect. The production switcher maybe used to create an effect involving multiple input video signals, e.g.a wipe or dissolve from a first input video signal to a second inputvideo signal. The production switcher may also be used for otherpurposes, for example to apply a colored border to the image representedby the input video signal. The digital video effects device may be usedto create an effect involving a change in size of the image representedby the input video signal relative to the raster of the output videosignal, or a change in position of the image represented by the inputvideo signal relative to the output raster.

These video processing machines are highly complex and have numerousfunctions. The status of each function is defined by the value of aparameter associated with the function. For example, in the case of aproduction switcher executing a dissolve from input video A to inputvideo B, appropriate functions might be

select video A as program input for mixer M1

select video B as preset input for mixer M1

mix coefficient for mixer M1.

The parameters associated with the first two functions would typicallybe binary in nature and would each be logical 1 throughout the effect,and the parameter associated with the third function might be a ten bitnumber that would change smoothly during the effect from logical 0,indicating 100% video A and 0% video B, to logical 1, indicating 0%video A and 100% video B. Each of the other parameters is set to adefault value, typically logical 0, such that the associated function isnot invoked.

In order to create a simple effect, the machine operator specifies astarting keyframe, an ending keyframe and the number of video frames orfields over which the effect is to take place. Each keyframe is acomplete description of the machine state. If, for example, the machinehas 100 functions, the status of each function is defined by a parameterand the starting keyframe contains the starting value for each of the100 function parameters and the ending keyframe contains the endingvalue for each parameter. When the effect is run, the entire state ofthe machine is updated at each video frame. For the functions that arein use during the effect, the updated value for the associated parameteris obtained by interpolating between the parameter's starting value andthe parameter's ending value. Where the starting value and ending valueare the same (select video A and select video B in the case of theexample), the interpolated value is the same as the starting value.Similarly, for a function that is not invoked for the effect, the valueof the parameter is updated with the default value on each frame.

A somewhat more complex effect might involve translation of the imagerepresented by the input video signal from a starting position at timet₀ through three intermediate positions (times t₁, t₂ and t₃) to anending position (time t₄). In this case, it is necessary to specify fivekeyframes giving the position of the input image at each of the times t₀-t₄.

This approach to controlling operation of a video processing machine onthe basis of keyframes is straightforward and easy to understand, but ithas certain limitations in its application. For example, if the producerof the effect employing five keyframes also wishes to create a pictureborder that turns slowly from blue to red over the duration of theentire effect from t₀ to t₄, there are only two time values (t₀ and t₄)at which the parameters that describe the color of the picture bordermust have specified values. Since the entire state of the machine ischanged at each video frame and the parameters are interpolated betweenconsecutive keyframes, it is necessary to include border colorinformation in the three intermediate keyframes for times t₁ -t₃. Theproducer must therefore estimate the values of the border colorparameters at points in time (t₁ -t₃) that are not related to the way inwhich the border color function is intended to operate. This increasesthe time spent in creating an effect and increases the chance that theeffect will not be what was expected, requiring adjustment of theintermediate keyframes and taking time to estimate other values of theborder color parameters.

SUMMARY OF THE INVENTION

According to a first aspect of the present invention there is provided amethod of creating a video effect employing a processing machine havinga plurality of functions, each of which has a status that is selectivelyvariable in response to change in value of an associated functionparameter, said method comprising selecting a set of functions, said setof functions being fewer than said plurality of functions, specifying astarting keyframe containing a starting value for each of saidparameters, specifying an ending keyframe containing ending values foronly the parameters associated with the functions of said set,specifying a number of video frames, employing the starting keyframe toplace the machine in a starting state, in which each function has thestatus defined by the value of its associated parameter in the startingkeyframe, processing a first video frame with the processing machine inthe starting state, employing the starting keyframe, the ending keyframeand said number of video frames to calculate a next frame value for eachof the parameters associated with a function in said set, employing thenext frame values to place the machine in a next frame state, in whicheach function of the set has the status defined by the next frame valueof its associated parameter, while maintaining unaltered the values ofthe parameters associated with functions that are not members of saidset, and processing a next video frame with the machine in said nextframe state.

According to a second aspect of the present invention there is provideda method of creating a video effect employing a processing machinehaving a plurality of functions, each of which has a status that isselectively variable in response to change in value of an associatedfunction parameter, said method comprising selecting first and secondsets of functions, each set containing fewer functions than saidplurality of functions, specifying a starting keyframe containing astarting value for each of said parameters, specifying an intermediatekeyframe containing ending values for only the parameters associatedwith the functions of said first set, specifying an ending keyframecontaining ending values for only the parameters associated with thefunctions of said second set, specifying first and second numbers ofvideo frames, said second number being greater than said first number,employing the starting keyframe to place the machine in a startingstate, in which each function has the status defined by the value of itsassociated function parameter in the starting keyframe, processing afirst video frame with the processing machine in the starting state,employing the starting keyframe, the intermediate keyframe and the firstnumber to calculate a next frame value for each of the parametersassociated with a function in said first set, employing the startingkeyframe, the ending keyframe and the second number to calculate a nextframe value for each of the parameters associated with a function insaid second set and not in said first set, employing the next framevalues to place the machine in a next frame state, in which eachfunction of the first and second sets has the status defined by the nextframe value of its associated parameter, while maintaining unaltered thevalues of the parameters associated with functions that are not membersof the first set or the second set, and processing a next video framewith the machine in said next frame state.

BRIEF DESCRIPTION OF THE DRAWINGS

For a better understanding of the invention, and to show how the samemay be carried into effect, reference will now be made, by way ofexample, to the accompanying drawings, in which:

FIG. 1 is a block diagram of apparatus for creating video effects; and

FIG. 2 is a block diagram of a component of the apparatus.

DETAILED DESCRIPTION

The apparatus shown in FIG. 1 comprises a crosspoint switching matrix 10having multiple inputs connected to receive respective input videosignals and having multiple outputs. One set of outputs of thecrosspoint matrix 10 are connected to respective inputs of a productionswitcher 12 and another set of outputs of the crosspoint matrix areconnected to inputs of at least one digital video effects (DVE) device14. The DVE device 14 has a DVE program output 18 that is connected to afurther input of the production switcher. The switcher has a systemprogram output 20, and also has an auxiliary output 22 that is connectedto an input of the DVE device 14. The matrix 10 receives key signalsassociated with the respective input video signals, and the key signalsare supplied to the DVE device 14 and switcher 12, and the outputs ofthe DVE device 14 and switcher 12 are video signals and associated keysignals. The matrix 10, the switcher 12 and the DVE device 14 may all beof conventional form. It will be appreciated that the apparatus mightcomprise several DVE devices performing different functions.

The apparatus shown in FIG. 1 also has a console 30 having manualcontrols for setting the state of the production switcher 12 and the DVEdevice 14. Each manual control includes a transducer for generating anelectrical signal in response to actuation of the control. In the caseof an on-off switch, the signal is binary in nature, whereas in the caseof a lever arm or knob having a range of positions and typicallycalibrated by reference to an analog scale, the signal might be aten-bit wide digital signal. The signal provided by a transducer isrepresentative of the value of the parameter associated with thefunction that is controlled by the particular manual control.

The console 30 communicates with the production switcher 12 and the DVEdevice 14 by way of a controller 32 and a bus 34. The controller 32generates digital words representative of the values of the parametersand delivers these digital words to the switcher 12 and the DVE device14 by way of the bus 34.

The production switcher 12 and the DVE device 14 each include multipleprocessing operators. A typical processing operator 36 is illustrated ingeneric form in FIG. 2, and has at least one video signal input, a videosignal output and a parameter input. The digital word representing thevalue of the parameter associated with the function controlled by atransducer is loaded from the bus 34 into a latch 38, whose output isconnected to the parameter input of the processing operator 36. Theprocessing operator operates on the video input in accordance with afunction that depends on the nature of the operator and has an argumentthat depends on the control signal. For example, in the switcher 12 theoperator might be a mixer that receives two input video signals A and Band a control signal X and returns an output signal A*X+B*(1-X).

Similarly, the DVE device performs multiple functions such as scaling,translation and rotation. The functions of the DVE device generallydepend upon the relationship between read and write addresses used toaccess a frame or field memory, and consequently it is not accurate todiscuss these functions as discrete operations. Nevertheless, the DVEdevice includes an interface that receives the parameter (scale, rotate,translate) signals from the console and generates appropriate controlsignals for application to the addressing circuitry of the DVE device.Further, the DVE device may include an input selector and one or moremixers, and such operators would receive suitable control signalsprovided by the console.

In order to construct an effect, the producer determines the length ofthe effect (usually in video frames) and describes what is to happenduring selected intervals within the effect. This might be accomplishedby sketching the effect at various key points, or writing a narrativeaccount referring to the key points. The producer specifies at least twokeyframes. For the purpose of the following example, it will be assumedthat the effect involves a background scene with a blue border thatprogressively changes to red over the duration of the effect. During theeffect, a foreground scene derived from video A is moved from an initiallocation to a final location by way of three intermediate locations,without any change in scale or rotation. Initially, the background sceneis derived from video B, but while the foreground scene moves from thefirst intermediate location to the final location, the background videochanges linearly to video C. No drop shadow effect or mirror effect isinvoked.

In order to create this effect, the producer first initializes theapparatus by clearing all functions to the off state. The producer thenspecifies five keyframes (KF₀ -KF₄) by operating the apparatus in a"create effect" mode. In this mode, the producer first selects KF₀,specifies the starting value for each parameter by reference to theeffect description, and then selects INSERT to load the parameter valuesinto a random access memory (RAM) associated with the controller 32. Asin the case of the known video processing machine, keyframe 0 is acomplete description of the machine's initial state. The contents ofkeyframe 0 are indicated in column 0 of Table I. After entering theparameter values for keyframe 0, the operator selects keyframe 1, andspecifies the values of parameters that are critical at the time ofkeyframe 1. In the case of the example, the description specifies thatthe foreground video should be at location (X₁, Y₁) in the raster of theoutput video signal and that a mix from video B to video C shouldcommence. The operator therefore specifies the values indicated incolumn 1 of Table I. Values for other parameters are not specified. Thecontroller determines what parameter values have been changed either bydynamically tracking the state of the manual controls that the operatorchanges during set-up of keyframe 1 or by testing the setting of eachcontrol against the value of the associated parameter stored forkeyframe 0. The operator selects INSERT and loads the parameter valuesfor keyframe 1. In similar fashion, parameter values for keyframes 2, 3and 4 are entered.

                  TABLE I                                                         ______________________________________                                                 Keyframe                                                                        0        1       2      3     4                                    ______________________________________                                        Insert FGD on                                                                 Foreground on                                                                 Select Video A                                                                           on                                                                 Select Video B                                                                           off                                                                Select Video C                                                                           off                                                                Select Mixer M1                                                                          off                                                                Background on                                                                 Select Video A                                                                           off                                                                Select Video B                                                                           on                                                                 Select Video C                                                                           off                                                                Select Mixer M1                                                                          off      on                                                        FGD Location                                                                             (X.sub.0,Y.sub.0)                                                                      (X.sub.1,Y.sub.1)                                                                     (X.sub.2,Y.sub.2)                                                                    (X.sub.3,Y.sub.3)                                                                   (X.sub.4,Y.sub.4)                    Border     on                                                                 Border width                                                                             W1                                                                 Border color                                                                             blue                          red                                  Mixer M1   off      on                                                        Select video A      off                                                       Select video B      on                                                        Select video C      on                                                        Mix coefficient     0%                   100%                                 Other effects                                                                            off                                                                ______________________________________                                    

In Table I, the entry other effects relates to other effects that mightbe used but are not in fact used in constructing the overall effect.

The parameter values associated with a given keyframe are stored in aregion of the RAM that is accessed by use of an address representativeof the keyframe number. For keyframes other than the first keyframe, thedata that is stored within the associated region of the memory isrepresentative only of the parameters that are active and for whichvalues are specified for that keyframe. For example, the contents of thememory regions for keyframes 0, 1 and 4 are shown in Table II.

                  TABLE II                                                        ______________________________________                                                   Keyframe 0                                                                              Keyframe 1    Keyframe 4                                 ______________________________________                                        Insert FGD on        Background    FGD                                                                           Location                                   Foreground           Select video B off                                                                          (X.sub.4,Y.sub.4)                          Select Video A                                                                           on        Select Mixer M1 on                                                                          Border                                     Select Video B                                                                           off       FGD Location  Color red                                  Select Video C                                                                           off       (X.sub.1,Y.sub.1)                                                                           Mixer M1                                   Select Mixer M1                                                                          off       Mixer M1      Coeff.                                                                        100%                                       Background           Select Video B on                                        Select Video A                                                                           off       Select Video C on                                        Select Video B                                                                           on        Coeff. 0%                                                Select Video C                                                                           off                                                                Select Mixer M1                                                                          off                                                                FGD Location                                                                             (X.sub.0,Y.sub.0)                                                  Border on                                                                     Width W1                                                                      Color Blue                                                                    Mixer M1 off                                                                  Other effects                                                                            off                                                                ______________________________________                                    

When the parameter values for the keyframes have been entered, theproducer operates the apparatus in the "run effect" mode. The valuesspecified in keyframe 0 are applied to the production switcher and theDVE device, and the apparatus processes the first video frame in thisstate. During processing of the first video frame, the controller teststhe value of each parameter for which the function is on in keyframe 0against each of the later keyframes to determine what, if any, is theearliest later keyframe at which another value of that parameter isspecified. For each parameter that is identified in this test, thecontroller calculates an interpolated value for use during processing ofthe second video frame. The relationship between the interpolated valueand the source (KF₀) and destination (KF₁) values for the parameterdepends on the interpolation function that is associated with theparameter. In the case of a parameter that can vary in aquasi-continuous fashion, linear interpolation, S-linear interpolationor cubic interpolation might be employed. In the case of a binaryparameter, the interpolation function would normally be a zero order orhold function. For the location parameter, the controller finds that thevalue (X₁,Y₁) is specified for keyframe 1. Since location is aquasi-continuous variable, the controller calculates an interpolatedvalue for location based on the values specified in keyframe 0 andkeyframe 1 and the number of video frames between keyframe 0 andkeyframe 1 and delivers this value of the location parameter to theDVE's addressing circuitry. For the border color parameter, thecontroller determines that red is specified for keyframe 4, andcalculates an updated color from the parameters defining blue and redand the number of video frames between keyframe 0 and keyframe 4.Functions for which the parameter is specified in keyframe 0 as beingoff are ignored, as are parameters that are not off but for which thereis no other value specified for any of the later keyframes. During thevertical blanking interval following the second field of video frame 0,the updated values for location and border color are applied to the DVEdevice and the production switcher respectively, whereas the values forthe other functions remain unchanged. The values for the other functionsare not reloaded, but remain unchanged. This operation is repeated forsuccessive video frames until processing frame N₁ -1, where N₁ is thenumber of the video frame associated with keyframe 1. During frame N₁-1, the controller selects the values for location, status of mixer M1,select video B, select video C and mix coefficient that are specifiedfor keyframe 1, but calculates interpolated color parameter values forthe border color function. These values are applied to the appropriateoperators and are used to process frame N₁. During frames N₁ to N₂, theoperation proceeds as before except that the location value isinterpolated between (X₁, Y₁) and (X₂, Y₂), mixer M1 is operative andthe mix coefficient is interpolated between 0% at frame N₁ and 100% atframe N₄. From frame N₂ until frame N₄, the operation continues insimilar fashion.

By defining the keyframes that occur after KF₀ as including onlyparameters that are given new values at the video frame with which thekeyframe is associated, it is not necessary for the operator to estimateor calculate values that are not related to the change that is to takeplace at that video frame. Therefore, it takes less time to create theeffect than with conventional machines, and the possibility of theeffect differing from what was expected is reduced.

The topology of the controller 32 is essentially the same as thetopology of the controller of a convention video processing machine, inthat it includes a processor, a program memory and a working memory, butthe contents of the program memory support the method described withreference to FIGS. 1 and 2.

It will be appreciated that the invention is not restricted to theparticular embodiment that has been described, and that variationstherein may be made without departing from the scope of the invention asdefined in the appended claims and equivalents thereof. For example,although the invention has been described in the context of a videoeffect in which the time granularity is one video frame, it could alsobe applied to an effect in which the time granularity is one videofield. Accordingly, video frame as used in the following claims meansboth a video frame (either interlaced or non-interlaced) and a videofield. Also, it is not necessary that the region of RAM associated witha particular keyframe contain no data relating to a parameter for whichno value is specified for that keyframe, and the RAM may instead containan entry confirming that the parameter has no value specified for thekeyframe.

It is claimed:
 1. A method of creating a video effect using a processingmachine having a plurality of functions, each function having at leastone parameter that defines either a state or a variable, the videoeffect further being defined by a sequence of keyframes, comprising thesteps of:creating a keyframe table defining a start keyframe, an endkeyframe and at least one intermediate keyframe, the start keyframehaving a start parameter value for each function of the processingmachine to be used for the video effect, and each subsequent keyframehaving a next parameter value only for those parameters that arecritical at that keyframe; setting an interval between each keyframe inthe sequence; and interpolating the parameter values of the variableparameters for video frames between keyframes as a function of thedifference between start and next parameter values and a total interval,the total interval being the sum of the intervals between the keyframescontaining the start and next parameter values.
 2. The method as recitedin claim 1 wherein the interpolating step comprises the step of linearinterpolating between the parameter values.
 3. The method as recited inclaim 1 wherein the interpolating step comprises the step of S-linearinterpolating between the parameter values.
 4. The method as recited inclaim 1 wherein the interpolating step comprises the step of cubicinterpolating between the parameter values.
 5. The method as recited inclaim 1 wherein the interpolating step comprises the step of calculatingbetween parameter values by a zero order function.